Skip to content

BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs - #53515

Merged
LucasWilkinson merged 10 commits into
vllm-project:mainfrom
pisceskkk:codex/pcp-piecewise-debug-main
Aug 26, 2026
Merged

LucasWilkinson merged 10 commits into
vllm-project:mainfrom
pisceskkk:codex/pcp-piecewise-debug-main

Conversation

@pisceskkk

@pisceskkk pisceskkk commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix PIECEWISE CUDA graph execution with PCP by using persistent PCP input buffers during graph capture and sizing graph dispatch from the largest real rank-local PCP batch before padding.

Implementation

  • Decoder graph capture now uses PCPManager.input_buffers when PCP is active, ensuring capture and runtime partitioning share persistent input_ids, positions, and is_padding buffers.
  • GPUModelRunner obtains the graph dispatch size from PCPManager.get_num_tokens_for_dispatch().
  • The CUDA graph dispatcher selects the smallest captured token size covering that rank-local maximum.
  • The selected BatchExecutionDescriptor.num_tokens is passed to PCP partitioning as padded_num_tokens, which determines rank-local gather-index and mask capacity.
  • A shared _iter_rank_chunks() implements rank chunking for both dispatch-size calculation and segment construction.
  • PCP raises ValueError when padded_num_tokens is smaller than the largest rank-local batch.

Representative dispatch mappings:

Workload Global tokens Rank-local maximum Capture size
PCP2 prefill 8 4 4
PCP2 prefill 7 4 4
PCP2, 3 decodes 3 3 4
PCP2, 3 decodes + prefill 11 7 8
PCP4, 2 decodes + prefill 11 5 8

Validation configuration

  • NVIDIA H20 GPUs
  • GLM-4.7-Flash
  • TP1 + PCP4 + EP4
  • max_num_batched_tokens=1024
  • max_num_seqs=32
  • max_model_len=8192
  • gpu_memory_utilization=0.80
  • GSM8K 5-shot, temperature 0, seed 42, maximum output tokens 4096, concurrency 32
  • PR head: 1dfcbc289db596b38052c33606a6f32c09d65ee9
  • Validation used the image-aligned FlashMLA Python runtime file; the source changes under test were exactly the PR head.

All four ranks completed 11/11 PIECEWISE graph captures. The service remained healthy after the full run, and logs contained zero graph replay, CUDA error, input-address mismatch, illegal-memory, traceback, engine-dead, initialization-failure signatures.

Correctness results

Scope Revision Mode Correct Accuracy Invalid Errors Completion tokens
32-question smoke PR head Eager 24/32 75.000% 0 0 3,139
32-question smoke PR head PIECEWISE 25/32 78.125% 0 0 3,162
Full GSM8K PR head Eager 1085/1319 82.2593% 0 0 136,925
Full GSM8K PR head PIECEWISE 1073/1319 81.3495% 0 0 140,111
Full GSM8K Main baseline (d626108b1841888ec90aced33367149a6bbc7e4b) PIECEWISE 1037/1319 78.6202% 1 0 364,147

For the paired PR-head full runs, 50 questions were correct only in Eager, 38 only in PIECEWISE, and 1,188 produced the same prediction. The exact McNemar p-value was 0.24079312453900145.

Test coverage

  • pytest -q tests/v1/worker/test_gpu_pcp_manager.py: 8 passed, 1 warning in 7.72s
  • Covers PCP2 and PCP4 prefill, decode, mixed batches, replicated-decode PIECEWISE padding, the graph-capacity guard, and the PCP capture-buffer contract.
  • Ruff check, Ruff format check, compileall, and git diff --check passed.

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
@mergify mergify Bot added nvidia mrv2 Model Runner V2 specific labels Aug 24, 2026
@pisceskkk pisceskkk changed the title Fix PCP PIECEWISE cudagraph input buffers BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs Aug 24, 2026
@pisceskkk
pisceskkk marked this pull request as ready for review August 24, 2026 03:56

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@LucasWilkinson

Copy link
Copy Markdown
Contributor

I opened #53639 as a draft alternative for comparing a materially different dispatch order. This PR selects a CUDA graph from the global scheduled token count and preserves that width through PCP partitioning. #53639 instead computes the exact maximum PCP-local width before graph dispatch, keeps global input staging separate, and pads each rank only to the selected local graph bucket. For PCP=4 with a 1024-token pure prefill, that changes the per-rank graph width from 1024 to 256; replicated decode widths remain unchanged. Only one implementation should merge.

Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
@pisceskkk
pisceskkk force-pushed the codex/pcp-piecewise-debug-main branch from 1a6fa4a to 1dfcbc2 Compare August 25, 2026 07:11
@pisceskkk

Copy link
Copy Markdown
Contributor Author

I opened #53639 as a draft alternative for comparing a materially different dispatch order.

Dispatching with the actual tokens sharded by PCP is more beneficial. I’ve migrated some of the changes from #53639 and verified the accuracy.

Comment thread vllm/v1/worker/gpu/model_runner.py Outdated
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Comment thread vllm/v1/worker/gpu/model_runner.py Outdated
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

@LucasWilkinson LucasWilkinson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thank you! (sorry went ahead and applied the suggestions so i could kick off a CI run, let me know if you think they should be reverted)

@github-project-automation github-project-automation Bot moved this to Ready in NVIDIA Aug 25, 2026
@pisceskkk

Copy link
Copy Markdown
Contributor Author

LGTM thank you! (sorry went ahead and applied the suggestions so i could kick off a CI run, let me know if you think they should be reverted)

no problem, have fixed the CI/pre-commit and PTAL if the changes is what you want. Thanks for your review again!

@pisceskkk

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #85588 for commit 5149d19bea24.

@LucasWilkinson
LucasWilkinson enabled auto-merge (squash) August 26, 2026 04:52
@LucasWilkinson
LucasWilkinson merged commit b1fbbc2 into vllm-project:main Aug 26, 2026
96 checks passed
@github-project-automation github-project-automation Bot moved this from Ready to Done in NVIDIA Aug 26, 2026
LucasWilkinson added a commit to neuralmagic/vllm that referenced this pull request Aug 26, 2026
…llm-project#53515)

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
(cherry picked from commit b1fbbc2)
LucasWilkinson added a commit to neuralmagic/vllm that referenced this pull request Aug 26, 2026
…m-project#53427 merge

732f952 enabled PIECEWISE graphs for sparse MLA under PCP (with vllm-project#53515's
persistent input buffers); the vllm#53427 merge brought the old guard back.
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit c82aa7d)
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit ce62292)
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit a8aa573)
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit 48ac3cc)
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit 3cfe87e)
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
…n count

Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
(cherry picked from commit 1dfcbc2)
LopezCastroRoberto added a commit to LopezCastroRoberto/vllm that referenced this pull request Aug 31, 2026
…e from vllm-project#53515

Squash the final follow-up fixes from vLLM PR vllm-project#53515 so the inherited upstream change remains separate from our interoperability fixes.

Upstream-commits: 8e5fa50 8bdd185 5149d19

Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
am-cohere pushed a commit to am-cohere/vllm that referenced this pull request Sep 1, 2026
…llm-project#53515)

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
mikeshawcode pushed a commit to mikeshawcode/vllm that referenced this pull request Sep 1, 2026
…llm-project#53515)

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: mikeshawcode <michaelwshaw2@gmail.com>
mylibrar pushed a commit to tanyuqian/vllm that referenced this pull request Sep 3, 2026
…llm-project#53515)

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Tflowers-0129 pushed a commit to vllm-project/vllm-ascend that referenced this pull request Sep 7, 2026
### What this PR does / why we need it

This PR upgrades the verified vLLM main anchor from
[`ba07e4a48fc951300d97eb506217dd530583dea3`](vllm-project/vllm@ba07e4a)
to
[`e6bfe03ad73a3330cb427885aa90d97a12e1c704`](vllm-project/vllm@e6bfe03).
The exact upstream range is
[ba07e4a48...e6bfe03ad](vllm-project/vllm@ba07e4a...e6bfe03).

The branch is rebased onto the latest vllm-ascend `origin/main`. The
current revision adds two follow-ups on top of the reviewed source
mapping:

- **`e67948b9c` - drop the local `pr_test.yaml` tweak.** The earlier
"raise e2e timeout for `ready-all` partitions" change is reverted, so
this PR no longer modifies `.github/workflows/pr_test.yaml`.
- **`fe7f550b7` - drop the vLLM `v0.27.1` release lane from the tests.**
Every `vllm_version_is("0.27.1")` gate in the unit/e2e suite is removed
and each site resolves to the vLLM-main behavior: dual-lane
if/else/ternary branches collapse to the main path, the v0.27.1-only
skip and the
`test_kimi_k3_gqa_mixed_groups_use_expected_physical_layout` test are
deleted, the profiling-time and `prepare_inputs` AST contract tests are
reshaped to the single main-lane implementation, dead `vllm_version_is`
mocks/imports are dropped, the e2e `hunyuan-vl` case always skips, the
obsolete `VLLM_VERSION=0.27.1` hack in `test_num_nans` is removed, and
the orphaned legacy `_get_kv_cache_config_deepseek_v4` planner is
deleted. `vllm_version_is()` stays in `vllm_ascend/utils.py` with its
unit test.

#### Review conclusion

- **Latest revision (`fe7f550b7`):** clean rebase onto current
`origin/main`; the two follow-ups above are committed and pushed.
Local-only, non-PR working-tree sources are not part of this branch.
- **Source review (earlier revisions):** the main2main adaptations for
the pinned upstream range were reviewed and are documented below; no
PR-introduced source-level blocker was found.
- **CI:** the run triggered on the rebased head (`fe7f550b7`) supersedes
the earlier run and is the authoritative gate for this revision.

#### Upstream changes covered

| Upstream PR | Exact commit | Contract adopted here |
|---|---|---|
| [#50465](vllm-project/vllm#50465) |
[`d154d90d6c`](vllm-project/vllm@d154d90)
| Batch-sharded sampling and `skip_gather` |
| [#51718](vllm-project/vllm#51718) |
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e)
| Standardized KV-cache layout |
| [#52209](vllm-project/vllm#52209) |
[`b26039b09f`](vllm-project/vllm@b26039b)
| Custom routed-expert weight loading |
| [#52560](vllm-project/vllm#52560) |
[`2f55ef254c`](vllm-project/vllm@2f55ef2)
| Qwen3-Omni DSpark support |
| [#52816](vllm-project/vllm#52816) |
[`b389ac2946`](vllm-project/vllm@b389ac2)
| DFlash2 and DFlash class factories |
| [#53183](vllm-project/vllm#53183) |
[`4aab2b0ebe`](vllm-project/vllm@4aab2b0)
| MRV2 becomes the default runner |
| [#53435](vllm-project/vllm#53435) |
[`a9a17e7095`](vllm-project/vllm@a9a17e7)
| DFlash2 subclass loading fix |
| [#53508](vllm-project/vllm#53508) |
[`479eeb32d2`](vllm-project/vllm@479eeb3)
| Isolated sleep-mode KV allocations |
| [#53515](vllm-project/vllm#53515) |
[`b1fbbc2ade`](vllm-project/vllm@b1fbbc2)
| Persistent PCP graph input buffers |
| [#53694](vllm-project/vllm#53694) |
[`5acc1c4e4b`](vllm-project/vllm@5acc1c4)
| Spec-decode `dp_sync` contract |
| [#53869](vllm-project/vllm#53869) |
[`b3af042abd`](vllm-project/vllm@b3af042)
| PCP slot mappings for PIECEWISE capture |

### Changes by file

> Note: the per-file notes below document the reviewed source mapping
for the pinned upstream range. Where they describe code as keeping a
v0.27.1 lane, the latest revision (`fe7f550b7`) removes the
`vllm_version_is("0.27.1")` gates from the unit/e2e tests listed below
and deletes the 0.27.1-only coverage; see "What this PR does".

#### Repository metadata and CI

##### `.github/vllm-main-verified.commit`

1. Updates the verified vLLM main SHA to
`e6bfe03ad73a3330cb427885aa90d97a12e1c704`.
- Upstream: [exact compare
range](vllm-project/vllm@ba07e4a...e6bfe03).
- Review: correct; this is the exact new anchor used by the source and
CI review.

##### `.github/workflows/pr_test.yaml`

This revision reverts the earlier local e2e-timeout tweak; this PR no
longer modifies `.github/workflows/pr_test.yaml`.

#### Runtime source

##### `vllm_ascend/_310p/model_runner_310p.py`

1. Adds a version-aware `KVCacheTensor` layer-name accessor and keeps
v0.27.1 aliasing while allocating main-lane attention/Mamba buffers per
layer.
2. Marks the 310P runner as not supporting the standardized shared
backing and derives cache sizes from each layer spec.
- Upstream: [#51718](vllm-project/vllm#51718) /
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e).
- Review: correct; it avoids treating an all-layer descriptor size as
one layer's allocation.

##### `vllm_ascend/_310p/worker/v2/model_runner.py`

1. Reads `shared_by` on v0.27.1 and `layers` on main when binding 310P
V2 KV tensors.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the active descriptor field is selected without
changing the release-lane behavior.

##### `vllm_ascend/_310p/worker_310p.py`

1. Applies the multi-group KV-memory scaling helper when the runner
cannot consume standardized shared backing.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this prevents per-layer materialization from
exceeding the planner's shared-allocation budget.

##### `vllm_ascend/attention/context_parallel/dsa_cp.py`

1. Reads the DeepSeek V4 compression ratio from `compress_ratio` on
v0.27.1 or `tokens_per_state` on main.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; both fields encode the same logical ratio in their
respective lanes.

##### `vllm_ascend/attention/dsa_v1.py`

1. Applies the same `compress_ratio` / `tokens_per_state` compatibility
when building DSA metadata.
2. Retains `AscendDSABackend.get_kv_cache_shape` intentionally: main
removed the generic base declaration, but Ascend allocation code still
calls the concrete backend helper.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; deleting the concrete helper would break Ascend's own
allocator.

##### `vllm_ascend/core/kv_cache_interface.py`

1. Makes `AscendMLAAttentionSpec.storage_block_size` and `merge()`
lane-aware for `compress_ratio` versus `tokens_per_state`.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; layout compatibility is compared using the field that
exists in each lane.

##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py`

1. Replaces direct `shared_by` reads with the version-aware helper.
2. Registers each real per-layer storage when one standardized
descriptor represents multiple private Ascend buffers.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; registration uses actual aligned storage addresses
instead of assuming descriptor-level aliasing.

#####
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py`

1. Uses the version-aware tensor-layer accessor for hybrid Mooncake
transfers.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py`

1. Uses the version-aware tensor-layer accessor for layerwise Mooncake
transfers.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py`

1. Reads layer names through the compatibility helper.
2. Constructs v0.27.1 tensors with `shared_by` and main tensors with
`layers`, `layer_stride`, `block_stride`, and `offset`.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the produced descriptor is valid in both dataclass
versions.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/offloading_connector.py`

1. Removes the deleted `is_kv_cache_tensor_packed` import/call and uses
`bool(block_stride)` on main.
2. Replaces `shared_by` with the version-aware layer accessor.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this resolves both introduced P1 import/call findings
while preserving the old packed-layout meaning.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py`

1. Uses the version-aware layer accessor when building recompute offload
metadata.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/worker.py`

1. Preserves new descriptor geometry (`layers`, strides, offset) on main
and old `shared_by` construction on v0.27.1.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; reconstructed tensors retain the layout information
required by main.

##### `vllm_ascend/models/deepseek_v4/indexer.py`

1. Constructs `AscendMLAAttentionSpec` with `compress_ratio` on v0.27.1
and `tokens_per_state` on main.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

##### `vllm_ascend/models/layer/attention/layer.py`

1. Applies the same lane-specific MLA spec field when attention layers
publish their KV-cache specs.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

##### `vllm_ascend/models/qwen3_dflash2.py`

1. Declares `decoder_layer_cls` and `model_cls` for the new upstream
factory-based construction path.
2. Retains the module-global swap only for v0.27.1, where the factories
do not exist.
- Upstream: [#52816](vllm-project/vllm#52816) /
[`b389ac2946`](vllm-project/vllm@b389ac2),
finalized by [#53435](vllm-project/vllm#53435) /
[`a9a17e7095`](vllm-project/vllm@a9a17e7).
- Review: correct; each lane instantiates `DFlash2Qwen3DecoderLayer` and
`DFlash2Qwen3Model` through its native mechanism.

##### `vllm_ascend/ops/vocab_parallel_embedding.py`

1. Adds the new `skip_gather` argument and mirrors the upstream early
return before tensor-parallel gather.
- Upstream: [#50465](vllm-project/vllm#50465) /
[`d154d90d6c`](vllm-project/vllm@d154d90).
- Review: correct; the trailing default keeps the old call contract
valid.

##### `vllm_ascend/patch/platform/patch_fused_moe.py`

1. Composes an upstream custom `RoutedExperts` subclass with
`AscendRoutedExperts` instead of replacing the class by name.
2. Preserves the upstream subclass's custom loader while retaining
Ascend routing/EPLB behavior.
- Upstream: [#52209](vllm-project/vllm#52209) /
[`b26039b09f`](vllm-project/vllm@b26039b).
- Review: correct; it adapts the actual factory return type and avoids
bypassing new upstream loading behavior.

##### `vllm_ascend/patch/platform/patch_kv_cache_utils.py`

1. Constructs lane-correct `KVCacheTensor` descriptors and inlines
page-size calculation removed from the old patch target.
2. Replaces the removed `_get_kv_cache_config_packed` hook on main with
patches for `get_kv_cache_config_from_groups`,
`_max_memory_usage_bytes_from_groups`, and `_pool_bytes_per_block`.
3. Preserves DeepSeek V4 shared tuples and rank-consistent KV block
planning.
- Upstream: [#51718](vllm-project/vllm#51718) /
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e).
- Review: correct; this resolves the introduced P0 removed-target
finding against the live main entry points.

##### `vllm_ascend/patch/platform/patch_use_v2_model_runner.py`

1. Removes DSpark and DFlash2 from Ascend's V1-only unsupported-feature
result when the upstream helper exists.
- Upstream: MRV2 default switch
[#53183](vllm-project/vllm#53183), with DSpark
from [#52560](vllm-project/vllm#52560) and
DFlash2 from [#52816](vllm-project/vllm#52816).
- Review: correct; the filter is narrow and does not change other
unsupported features.

##### `vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py`

1. Keeps the legacy `_allocate_kv_cache` / `_reshape_kv_cache` patches
only on v0.27.1.
2. Patches main's live `allocate_kv_cache` entry point with
`allocate_kv_cache_main` and retains Ascend reshape binding.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this resolves the removed-import and
removed-monkey-patch-target P0/P1 findings.

##### `vllm_ascend/utils.py`

1. Adds `get_kv_cache_tensor_layers()` to normalize `shared_by` and
`layers` reads.
- Upstream: [#51718](vllm-project/vllm#51718).
2. Strips a PEP 440 local suffix (for example `+empty`) before
`vllm_version_is()` comparison.
- Upstream: no direct upstream patch; downstream compatibility hardening
needed for release-lane version strings.
- Review: correct; the comparison changes only local build metadata
handling.

##### `vllm_ascend/worker/model_runner_v1.py`

1. Implements lane-correct KV descriptor reads and advertises support
for standardized shared backing.
2. On main, overlays compatible attention/Mamba groups in one backing
store and exposes descriptor-offset views; otherwise materializes
correctly sized private per-layer buffers.
3. Preserves v0.27.1 aliasing, SFA/indexer layouts, sparse/offload
paths, cache-only caches, and page-padding geometry.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the allocation follows the new descriptor geometry
without changing the release-lane memory model.

##### `vllm_ascend/worker/v2/aclgraph_utils.py`

1. Keeps old-lane dummy-batch repartitioning but consumes already-local
persistent buffers on main.
2. Uses PCP dummy block tables and slot mappings and forwards
`pcp_manager` through graph capture.
- Upstream: persistent buffers
[#53515](vllm-project/vllm#53515) and capture
slot mappings [#53869](vllm-project/vllm#53869).
- Review: correct; main no longer repartitions an already rank-local
capture batch.

##### `vllm_ascend/worker/v2/attn_utils.py`

1. Removes main-lane dependence on the deleted
`indexes_kv_by_block_stride` marker and uses standardized page geometry.
2. Allocates one hybrid backing on main, then creates per-layer views
from `offset`, `layer_stride`, and `block_stride`; private SFA/attention
allocations are retained where sharing is invalid.
3. Adds `allocate_kv_cache_main`, reconstructs Ascend attention groups,
and binds the live upstream allocation entry point.
4. Retains calls to concrete Ascend `get_kv_cache_shape` helpers because
Ascend still needs backend-specific views after the generic base method
was removed.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the three machine-reported calls are intentional
concrete-backend calls, not calls to the removed base implementation.

##### `vllm_ascend/worker/v2/model_runner.py`

1. Advertises standardized shared KV backing and keeps separate
`prepare_inputs` implementations for the two upstream signatures.
2. Preserves the larger of real PCP tokens and graph-descriptor padding;
main forwards `padded_num_tokens` to the PCP manager.
- Upstream: KV layout
[#51718](vllm-project/vllm#51718), persistent
PCP buffers [#53515](vllm-project/vllm#53515),
and capture mappings
[#53869](vllm-project/vllm#53869).
- Review: correct; runtime PCP tokens are not truncated to the graph
descriptor.

##### `vllm_ascend/worker/v2/pcp_manager.py`

1. Matches the optional constructor/partition keywords exposed by each
lane.
2. Uses persistent `AscendInputBuffers`, including the `max_num_reqs +
1` query-offset view required by prefix sums.
3. Preserves explicit graph padding in the main-lane local batch.
- Upstream: [#53515](vllm-project/vllm#53515)
and [#53869](vllm-project/vllm#53869).
- Review: correct; buffer lifetime, shape, and padding match the new PCP
capture contract.

##### `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py`

1. Accepts `dp_sync`, forwards `num_tokens_across_dp` on v0.27.1, and
forwards `dp_sync` on main.
- Upstream: [#53694](vllm-project/vllm#53694) /
[`5acc1c4e4b`](vllm-project/vllm@5acc1c4).
   - Review: correct.

##### `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py`

1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync`
forwarding in DFlash.
- Upstream: [#53694](vllm-project/vllm#53694).
   - Review: correct.

##### `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py`

1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync`
forwarding in DSpark.
- Upstream: [#53694](vllm-project/vllm#53694).
   - Review: correct.

##### `vllm_ascend/worker/worker.py`

1. Guards the removed `post_kv_cache_wake_up` hook with `hasattr`.
- Upstream: [#53508](vllm-project/vllm#53508) /
[`479eeb32d2`](vllm-project/vllm@479eeb3).
2. Scales multi-group KV memory only when Ascend must materialize
private buffers; skips DeepSeek V4 custom planning and compatible
standardized hybrid sharing.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; it prevents OOM without reducing capacity for runners
that can consume the shared layout.

#### Unit tests

##### `tests/ut/_310p/test_model_runner_310p.py`

1. Verifies that the 310P runner does not advertise standardized shared
KV backing.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/_310p/test_model_runner_v2_310p.py`

1. Makes the descriptor fixture valid with `shared_by` on v0.27.1 and
`layers` on main.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/attention/test_dsa_v1.py`

1. Covers both `compress_ratio` and `tokens_per_state` metadata inputs.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/core/test_recompute_scheduler.py`

1. Constructs base MLA specs with the ratio field available in the
active lane.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py`

1. Adds a lane-aware `KVCacheTensor` fixture and validates layout reads
through the compatibility helper.
2. Covers main descriptor strides/offsets and packed-descriptor
rejection.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/kv_offload/test_mooncake_connector.py`

1. Adapts fixtures/assertions to standardized descriptors and main
group-allocation sizes.
2. Adds coverage for registering multiple private per-layer storages
represented by one descriptor.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/patch/platform/test_patch_fused_moe.py`

1. Verifies composition of a custom upstream routed-expert subclass with
the Ascend contract and loader preservation.
- Upstream coverage:
[#52209](vllm-project/vllm#52209).

##### `tests/ut/patch/platform/test_patch_use_v2_model_runner.py`

1. Verifies that only DSpark and DFlash2 are removed from the V1-only
unsupported list.
- Upstream coverage:
[#53183](vllm-project/vllm#53183),
[#52560](vllm-project/vllm#52560), and
[#52816](vllm-project/vllm#52816).

##### `tests/ut/patch/platform/test_prefix_cache_cp_patches.py`

1. Adapts standardized KV descriptor fixtures.
2. Covers DeepSeek V4 shared-tuple capacity and rank-consistent
replanning.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/spec_decode/test_dflash2_proposer.py`

1. Verifies DFlash2's `decoder_layer_cls` and `model_cls` declarations.
- Upstream coverage:
[#52816](vllm-project/vllm#52816) and
[#53435](vllm-project/vllm#53435).

##### `tests/ut/test_compressed_prefix_cache.py`

1. Constructs compressed-prefix MLA specs with the lane-specific ratio
field.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/tools/bisect/test_version_compat.py`

1. Isolates `VLLM_VERSION` environment state so one compatibility test
cannot leak its lane into another.
- Upstream coverage: no direct source patch; downstream test isolation
for the two-lane compatibility logic.

##### `tests/ut/worker/a2/test_model_runner_v1.py`

1. Covers standardized descriptor allocation, per-layer views, shared
capacity, and cache-only behavior.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/a2/test_model_runner_v1_with_device.py`

1. Adapts device-backed V1 fixtures to `shared_by` / `layers` and
validates main-lane geometry.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/a2/test_worker_v1.py`

1. Covers shared-layout capacity versus private-buffer budget scaling.
2. Covers the optional wake hook after sleep-mode KV allocation changes.
- Upstream coverage:
[#51718](vllm-project/vllm#51718) and
[#53508](vllm-project/vllm#53508).

##### `tests/ut/worker/test_attn_utils_v2.py`

1. Covers the main `allocate_kv_cache` entry point, new descriptor
geometry, MLA ratio field, and flat attention-group reshape contract.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/test_model_runner_v2.py`

1. Verifies both prepare-input implementations preserve real PCP tokens
and only main forwards graph padding.
- Upstream coverage:
[#53515](vllm-project/vllm#53515) and
[#53869](vllm-project/vllm#53869).

##### `tests/ut/worker/test_model_runner_v2_finegrained_tp.py`

1. Adds `batch_sharder` and request-count fields to the bare fixture to
match the new sampling contract.
- Upstream coverage:
[#50465](vllm-project/vllm#50465).

##### `tests/ut/worker/test_model_runner_v2_mamba.py`

1. Adds lane-aware descriptors and validates one main-lane hybrid
backing with per-layer offsets.
2. Covers the removal of `indexes_kv_by_block_stride` through observable
page-padding geometry.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/test_pcp_manager_v2.py`

1. Covers persistent Ascend input buffers, prefix-sum offset shape,
padded local batches, main capture slot mappings, and old/new keyword
signatures.
2. Verifies `dp_sync` on all three Ascend speculators and `pcp_manager`
on graph capture.
- Upstream coverage:
[#53515](vllm-project/vllm#53515),
[#53694](vllm-project/vllm#53694), and
[#53869](vllm-project/vllm#53869).

### Does this PR introduce any user-facing change?

No. This is an internal compatibility update; it does not add an
Ascend-specific public API.

### How was this patch tested?

- Exact-contract main2main validation and range prediction for vLLM
[`ba07e4a48...e6bfe03ad`](vllm-project/vllm@ba07e4a...e6bfe03).
- GitHub Actions: [run
33229439657](https://github.com/vllm-project/vllm-ascend/actions/runs/33229439657).
- Successful in that run: pre-commit, both 310P jobs, all A3 jobs, and
the passing A2 shards on both `e6bfe03ad...` and `v0.27.1`.
- Pending rerun: ModelScope HTTP 500 failures on A2; two unchanged EPLB
CPU tests fail identically on both lanes.

- vLLM main:
vllm-project/vllm@ba07e4a

---------

Signed-off-by: liaoqidan <1107297340@qq.com>
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Signed-off-by: LQDLove <LQDLove@users.noreply.github.com>
Co-authored-by: shenzhao <shenzhao9@huawei.com>
Co-authored-by: LQDLove <LQDLove@users.noreply.github.com>
LopezCastroRoberto pushed a commit to LopezCastroRoberto/vllm that referenced this pull request Sep 7, 2026
… PIECEWISE CUDA graphs

Squash the complete validated vllm-project#53515 change set, including the final padding and rank-local dispatch updates, into one provenance commit.
LucasWilkinson pushed a commit to LucasWilkinson/vllm that referenced this pull request Sep 8, 2026
… PIECEWISE CUDA graphs

Squash the complete validated vllm-project#53515 change set, including the final padding and rank-local dispatch updates, into one provenance commit.
sunny-rain-63 pushed a commit to sunny-rain-63/vllm-ascend that referenced this pull request Sep 12, 2026
### What this PR does / why we need it

This PR upgrades the verified vLLM main anchor from
[`ba07e4a48fc951300d97eb506217dd530583dea3`](vllm-project/vllm@ba07e4a)
to
[`e6bfe03ad73a3330cb427885aa90d97a12e1c704`](vllm-project/vllm@e6bfe03).
The exact upstream range is
[ba07e4a48...e6bfe03ad](vllm-project/vllm@ba07e4a...e6bfe03).

The branch is rebased onto the latest vllm-ascend `origin/main`. The
current revision adds two follow-ups on top of the reviewed source
mapping:

- **`e67948b9c` - drop the local `pr_test.yaml` tweak.** The earlier
"raise e2e timeout for `ready-all` partitions" change is reverted, so
this PR no longer modifies `.github/workflows/pr_test.yaml`.
- **`fe7f550b7` - drop the vLLM `v0.27.1` release lane from the tests.**
Every `vllm_version_is("0.27.1")` gate in the unit/e2e suite is removed
and each site resolves to the vLLM-main behavior: dual-lane
if/else/ternary branches collapse to the main path, the v0.27.1-only
skip and the
`test_kimi_k3_gqa_mixed_groups_use_expected_physical_layout` test are
deleted, the profiling-time and `prepare_inputs` AST contract tests are
reshaped to the single main-lane implementation, dead `vllm_version_is`
mocks/imports are dropped, the e2e `hunyuan-vl` case always skips, the
obsolete `VLLM_VERSION=0.27.1` hack in `test_num_nans` is removed, and
the orphaned legacy `_get_kv_cache_config_deepseek_v4` planner is
deleted. `vllm_version_is()` stays in `vllm_ascend/utils.py` with its
unit test.

#### Review conclusion

- **Latest revision (`fe7f550b7`):** clean rebase onto current
`origin/main`; the two follow-ups above are committed and pushed.
Local-only, non-PR working-tree sources are not part of this branch.
- **Source review (earlier revisions):** the main2main adaptations for
the pinned upstream range were reviewed and are documented below; no
PR-introduced source-level blocker was found.
- **CI:** the run triggered on the rebased head (`fe7f550b7`) supersedes
the earlier run and is the authoritative gate for this revision.

#### Upstream changes covered

| Upstream PR | Exact commit | Contract adopted here |
|---|---|---|
| [#50465](vllm-project/vllm#50465) |
[`d154d90d6c`](vllm-project/vllm@d154d90)
| Batch-sharded sampling and `skip_gather` |
| [#51718](vllm-project/vllm#51718) |
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e)
| Standardized KV-cache layout |
| [#52209](vllm-project/vllm#52209) |
[`b26039b09f`](vllm-project/vllm@b26039b)
| Custom routed-expert weight loading |
| [#52560](vllm-project/vllm#52560) |
[`2f55ef254c`](vllm-project/vllm@2f55ef2)
| Qwen3-Omni DSpark support |
| [#52816](vllm-project/vllm#52816) |
[`b389ac2946`](vllm-project/vllm@b389ac2)
| DFlash2 and DFlash class factories |
| [#53183](vllm-project/vllm#53183) |
[`4aab2b0ebe`](vllm-project/vllm@4aab2b0)
| MRV2 becomes the default runner |
| [#53435](vllm-project/vllm#53435) |
[`a9a17e7095`](vllm-project/vllm@a9a17e7)
| DFlash2 subclass loading fix |
| [#53508](vllm-project/vllm#53508) |
[`479eeb32d2`](vllm-project/vllm@479eeb3)
| Isolated sleep-mode KV allocations |
| [#53515](vllm-project/vllm#53515) |
[`b1fbbc2ade`](vllm-project/vllm@b1fbbc2)
| Persistent PCP graph input buffers |
| [#53694](vllm-project/vllm#53694) |
[`5acc1c4e4b`](vllm-project/vllm@5acc1c4)
| Spec-decode `dp_sync` contract |
| [#53869](vllm-project/vllm#53869) |
[`b3af042abd`](vllm-project/vllm@b3af042)
| PCP slot mappings for PIECEWISE capture |

### Changes by file

> Note: the per-file notes below document the reviewed source mapping
for the pinned upstream range. Where they describe code as keeping a
v0.27.1 lane, the latest revision (`fe7f550b7`) removes the
`vllm_version_is("0.27.1")` gates from the unit/e2e tests listed below
and deletes the 0.27.1-only coverage; see "What this PR does".

#### Repository metadata and CI

##### `.github/vllm-main-verified.commit`

1. Updates the verified vLLM main SHA to
`e6bfe03ad73a3330cb427885aa90d97a12e1c704`.
- Upstream: [exact compare
range](vllm-project/vllm@ba07e4a...e6bfe03).
- Review: correct; this is the exact new anchor used by the source and
CI review.

##### `.github/workflows/pr_test.yaml`

This revision reverts the earlier local e2e-timeout tweak; this PR no
longer modifies `.github/workflows/pr_test.yaml`.

#### Runtime source

##### `vllm_ascend/_310p/model_runner_310p.py`

1. Adds a version-aware `KVCacheTensor` layer-name accessor and keeps
v0.27.1 aliasing while allocating main-lane attention/Mamba buffers per
layer.
2. Marks the 310P runner as not supporting the standardized shared
backing and derives cache sizes from each layer spec.
- Upstream: [#51718](vllm-project/vllm#51718) /
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e).
- Review: correct; it avoids treating an all-layer descriptor size as
one layer's allocation.

##### `vllm_ascend/_310p/worker/v2/model_runner.py`

1. Reads `shared_by` on v0.27.1 and `layers` on main when binding 310P
V2 KV tensors.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the active descriptor field is selected without
changing the release-lane behavior.

##### `vllm_ascend/_310p/worker_310p.py`

1. Applies the multi-group KV-memory scaling helper when the runner
cannot consume standardized shared backing.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this prevents per-layer materialization from
exceeding the planner's shared-allocation budget.

##### `vllm_ascend/attention/context_parallel/dsa_cp.py`

1. Reads the DeepSeek V4 compression ratio from `compress_ratio` on
v0.27.1 or `tokens_per_state` on main.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; both fields encode the same logical ratio in their
respective lanes.

##### `vllm_ascend/attention/dsa_v1.py`

1. Applies the same `compress_ratio` / `tokens_per_state` compatibility
when building DSA metadata.
2. Retains `AscendDSABackend.get_kv_cache_shape` intentionally: main
removed the generic base declaration, but Ascend allocation code still
calls the concrete backend helper.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; deleting the concrete helper would break Ascend's own
allocator.

##### `vllm_ascend/core/kv_cache_interface.py`

1. Makes `AscendMLAAttentionSpec.storage_block_size` and `merge()`
lane-aware for `compress_ratio` versus `tokens_per_state`.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; layout compatibility is compared using the field that
exists in each lane.

##### `vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_connector.py`

1. Replaces direct `shared_by` reads with the version-aware helper.
2. Registers each real per-layer storage when one standardized
descriptor represents multiple private Ascend buffers.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; registration uses actual aligned storage addresses
instead of assuming descriptor-level aliasing.

#####
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_hybrid_connector.py`

1. Uses the version-aware tensor-layer accessor for hybrid Mooncake
transfers.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_p2p/mooncake_layerwise_connector.py`

1. Uses the version-aware tensor-layer accessor for layerwise Mooncake
transfers.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/layerwise_cache_layout.py`

1. Reads layer names through the compatibility helper.
2. Constructs v0.27.1 tensors with `shared_by` and main tensors with
`layers`, `layer_stride`, `block_stride`, and `offset`.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the produced descriptor is valid in both dataclass
versions.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/kv_offload/native/offloading_connector.py`

1. Removes the deleted `is_kv_cache_tensor_packed` import/call and uses
`bool(block_stride)` on main.
2. Replaces `shared_by` with the version-aware layer accessor.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this resolves both introduced P1 import/call findings
while preserving the old packed-layout meaning.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py`

1. Uses the version-aware layer accessor when building recompute offload
metadata.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

#####
`vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/worker.py`

1. Preserves new descriptor geometry (`layers`, strides, offset) on main
and old `shared_by` construction on v0.27.1.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; reconstructed tensors retain the layout information
required by main.

##### `vllm_ascend/models/deepseek_v4/indexer.py`

1. Constructs `AscendMLAAttentionSpec` with `compress_ratio` on v0.27.1
and `tokens_per_state` on main.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

##### `vllm_ascend/models/layer/attention/layer.py`

1. Applies the same lane-specific MLA spec field when attention layers
publish their KV-cache specs.
- Upstream: [#51718](vllm-project/vllm#51718).
   - Review: correct.

##### `vllm_ascend/models/qwen3_dflash2.py`

1. Declares `decoder_layer_cls` and `model_cls` for the new upstream
factory-based construction path.
2. Retains the module-global swap only for v0.27.1, where the factories
do not exist.
- Upstream: [#52816](vllm-project/vllm#52816) /
[`b389ac2946`](vllm-project/vllm@b389ac2),
finalized by [#53435](vllm-project/vllm#53435) /
[`a9a17e7095`](vllm-project/vllm@a9a17e7).
- Review: correct; each lane instantiates `DFlash2Qwen3DecoderLayer` and
`DFlash2Qwen3Model` through its native mechanism.

##### `vllm_ascend/ops/vocab_parallel_embedding.py`

1. Adds the new `skip_gather` argument and mirrors the upstream early
return before tensor-parallel gather.
- Upstream: [#50465](vllm-project/vllm#50465) /
[`d154d90d6c`](vllm-project/vllm@d154d90).
- Review: correct; the trailing default keeps the old call contract
valid.

##### `vllm_ascend/patch/platform/patch_fused_moe.py`

1. Composes an upstream custom `RoutedExperts` subclass with
`AscendRoutedExperts` instead of replacing the class by name.
2. Preserves the upstream subclass's custom loader while retaining
Ascend routing/EPLB behavior.
- Upstream: [#52209](vllm-project/vllm#52209) /
[`b26039b09f`](vllm-project/vllm@b26039b).
- Review: correct; it adapts the actual factory return type and avoids
bypassing new upstream loading behavior.

##### `vllm_ascend/patch/platform/patch_kv_cache_utils.py`

1. Constructs lane-correct `KVCacheTensor` descriptors and inlines
page-size calculation removed from the old patch target.
2. Replaces the removed `_get_kv_cache_config_packed` hook on main with
patches for `get_kv_cache_config_from_groups`,
`_max_memory_usage_bytes_from_groups`, and `_pool_bytes_per_block`.
3. Preserves DeepSeek V4 shared tuples and rank-consistent KV block
planning.
- Upstream: [#51718](vllm-project/vllm#51718) /
[`8bdc70ec7b`](vllm-project/vllm@8bdc70e).
- Review: correct; this resolves the introduced P0 removed-target
finding against the live main entry points.

##### `vllm_ascend/patch/platform/patch_use_v2_model_runner.py`

1. Removes DSpark and DFlash2 from Ascend's V1-only unsupported-feature
result when the upstream helper exists.
- Upstream: MRV2 default switch
[#53183](vllm-project/vllm#53183), with DSpark
from [#52560](vllm-project/vllm#52560) and
DFlash2 from [#52816](vllm-project/vllm#52816).
- Review: correct; the filter is narrow and does not change other
unsupported features.

##### `vllm_ascend/patch/worker/patch_v2/patch_attn_utils.py`

1. Keeps the legacy `_allocate_kv_cache` / `_reshape_kv_cache` patches
only on v0.27.1.
2. Patches main's live `allocate_kv_cache` entry point with
`allocate_kv_cache_main` and retains Ascend reshape binding.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; this resolves the removed-import and
removed-monkey-patch-target P0/P1 findings.

##### `vllm_ascend/utils.py`

1. Adds `get_kv_cache_tensor_layers()` to normalize `shared_by` and
`layers` reads.
- Upstream: [#51718](vllm-project/vllm#51718).
2. Strips a PEP 440 local suffix (for example `+empty`) before
`vllm_version_is()` comparison.
- Upstream: no direct upstream patch; downstream compatibility hardening
needed for release-lane version strings.
- Review: correct; the comparison changes only local build metadata
handling.

##### `vllm_ascend/worker/model_runner_v1.py`

1. Implements lane-correct KV descriptor reads and advertises support
for standardized shared backing.
2. On main, overlays compatible attention/Mamba groups in one backing
store and exposes descriptor-offset views; otherwise materializes
correctly sized private per-layer buffers.
3. Preserves v0.27.1 aliasing, SFA/indexer layouts, sparse/offload
paths, cache-only caches, and page-padding geometry.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the allocation follows the new descriptor geometry
without changing the release-lane memory model.

##### `vllm_ascend/worker/v2/aclgraph_utils.py`

1. Keeps old-lane dummy-batch repartitioning but consumes already-local
persistent buffers on main.
2. Uses PCP dummy block tables and slot mappings and forwards
`pcp_manager` through graph capture.
- Upstream: persistent buffers
[#53515](vllm-project/vllm#53515) and capture
slot mappings [#53869](vllm-project/vllm#53869).
- Review: correct; main no longer repartitions an already rank-local
capture batch.

##### `vllm_ascend/worker/v2/attn_utils.py`

1. Removes main-lane dependence on the deleted
`indexes_kv_by_block_stride` marker and uses standardized page geometry.
2. Allocates one hybrid backing on main, then creates per-layer views
from `offset`, `layer_stride`, and `block_stride`; private SFA/attention
allocations are retained where sharing is invalid.
3. Adds `allocate_kv_cache_main`, reconstructs Ascend attention groups,
and binds the live upstream allocation entry point.
4. Retains calls to concrete Ascend `get_kv_cache_shape` helpers because
Ascend still needs backend-specific views after the generic base method
was removed.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; the three machine-reported calls are intentional
concrete-backend calls, not calls to the removed base implementation.

##### `vllm_ascend/worker/v2/model_runner.py`

1. Advertises standardized shared KV backing and keeps separate
`prepare_inputs` implementations for the two upstream signatures.
2. Preserves the larger of real PCP tokens and graph-descriptor padding;
main forwards `padded_num_tokens` to the PCP manager.
- Upstream: KV layout
[#51718](vllm-project/vllm#51718), persistent
PCP buffers [#53515](vllm-project/vllm#53515),
and capture mappings
[#53869](vllm-project/vllm#53869).
- Review: correct; runtime PCP tokens are not truncated to the graph
descriptor.

##### `vllm_ascend/worker/v2/pcp_manager.py`

1. Matches the optional constructor/partition keywords exposed by each
lane.
2. Uses persistent `AscendInputBuffers`, including the `max_num_reqs +
1` query-offset view required by prefix sums.
3. Preserves explicit graph padding in the main-lane local batch.
- Upstream: [#53515](vllm-project/vllm#53515)
and [#53869](vllm-project/vllm#53869).
- Review: correct; buffer lifetime, shape, and padding match the new PCP
capture contract.

##### `vllm_ascend/worker/v2/spec_decode/autoregressive/speculator.py`

1. Accepts `dp_sync`, forwards `num_tokens_across_dp` on v0.27.1, and
forwards `dp_sync` on main.
- Upstream: [#53694](vllm-project/vllm#53694) /
[`5acc1c4e4b`](vllm-project/vllm@5acc1c4).
   - Review: correct.

##### `vllm_ascend/worker/v2/spec_decode/dflash/speculator.py`

1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync`
forwarding in DFlash.
- Upstream: [#53694](vllm-project/vllm#53694).
   - Review: correct.

##### `vllm_ascend/worker/v2/spec_decode/dspark/speculator.py`

1. Applies the same lane-specific `num_tokens_across_dp` / `dp_sync`
forwarding in DSpark.
- Upstream: [#53694](vllm-project/vllm#53694).
   - Review: correct.

##### `vllm_ascend/worker/worker.py`

1. Guards the removed `post_kv_cache_wake_up` hook with `hasattr`.
- Upstream: [#53508](vllm-project/vllm#53508) /
[`479eeb32d2`](vllm-project/vllm@479eeb3).
2. Scales multi-group KV memory only when Ascend must materialize
private buffers; skips DeepSeek V4 custom planning and compatible
standardized hybrid sharing.
- Upstream: [#51718](vllm-project/vllm#51718).
- Review: correct; it prevents OOM without reducing capacity for runners
that can consume the shared layout.

#### Unit tests

##### `tests/ut/_310p/test_model_runner_310p.py`

1. Verifies that the 310P runner does not advertise standardized shared
KV backing.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/_310p/test_model_runner_v2_310p.py`

1. Makes the descriptor fixture valid with `shared_by` on v0.27.1 and
`layers` on main.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/attention/test_dsa_v1.py`

1. Covers both `compress_ratio` and `tokens_per_state` metadata inputs.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/core/test_recompute_scheduler.py`

1. Constructs base MLA specs with the ratio field available in the
active lane.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/distributed/ascend_store/test_layerwise_cache_layout.py`

1. Adds a lane-aware `KVCacheTensor` fixture and validates layout reads
through the compatibility helper.
2. Covers main descriptor strides/offsets and packed-descriptor
rejection.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/kv_offload/test_mooncake_connector.py`

1. Adapts fixtures/assertions to standardized descriptors and main
group-allocation sizes.
2. Adds coverage for registering multiple private per-layer storages
represented by one descriptor.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/patch/platform/test_patch_fused_moe.py`

1. Verifies composition of a custom upstream routed-expert subclass with
the Ascend contract and loader preservation.
- Upstream coverage:
[#52209](vllm-project/vllm#52209).

##### `tests/ut/patch/platform/test_patch_use_v2_model_runner.py`

1. Verifies that only DSpark and DFlash2 are removed from the V1-only
unsupported list.
- Upstream coverage:
[#53183](vllm-project/vllm#53183),
[#52560](vllm-project/vllm#52560), and
[#52816](vllm-project/vllm#52816).

##### `tests/ut/patch/platform/test_prefix_cache_cp_patches.py`

1. Adapts standardized KV descriptor fixtures.
2. Covers DeepSeek V4 shared-tuple capacity and rank-consistent
replanning.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/spec_decode/test_dflash2_proposer.py`

1. Verifies DFlash2's `decoder_layer_cls` and `model_cls` declarations.
- Upstream coverage:
[#52816](vllm-project/vllm#52816) and
[#53435](vllm-project/vllm#53435).

##### `tests/ut/test_compressed_prefix_cache.py`

1. Constructs compressed-prefix MLA specs with the lane-specific ratio
field.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/tools/bisect/test_version_compat.py`

1. Isolates `VLLM_VERSION` environment state so one compatibility test
cannot leak its lane into another.
- Upstream coverage: no direct source patch; downstream test isolation
for the two-lane compatibility logic.

##### `tests/ut/worker/a2/test_model_runner_v1.py`

1. Covers standardized descriptor allocation, per-layer views, shared
capacity, and cache-only behavior.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/a2/test_model_runner_v1_with_device.py`

1. Adapts device-backed V1 fixtures to `shared_by` / `layers` and
validates main-lane geometry.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/a2/test_worker_v1.py`

1. Covers shared-layout capacity versus private-buffer budget scaling.
2. Covers the optional wake hook after sleep-mode KV allocation changes.
- Upstream coverage:
[#51718](vllm-project/vllm#51718) and
[#53508](vllm-project/vllm#53508).

##### `tests/ut/worker/test_attn_utils_v2.py`

1. Covers the main `allocate_kv_cache` entry point, new descriptor
geometry, MLA ratio field, and flat attention-group reshape contract.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/test_model_runner_v2.py`

1. Verifies both prepare-input implementations preserve real PCP tokens
and only main forwards graph padding.
- Upstream coverage:
[#53515](vllm-project/vllm#53515) and
[#53869](vllm-project/vllm#53869).

##### `tests/ut/worker/test_model_runner_v2_finegrained_tp.py`

1. Adds `batch_sharder` and request-count fields to the bare fixture to
match the new sampling contract.
- Upstream coverage:
[#50465](vllm-project/vllm#50465).

##### `tests/ut/worker/test_model_runner_v2_mamba.py`

1. Adds lane-aware descriptors and validates one main-lane hybrid
backing with per-layer offsets.
2. Covers the removal of `indexes_kv_by_block_stride` through observable
page-padding geometry.
- Upstream coverage:
[#51718](vllm-project/vllm#51718).

##### `tests/ut/worker/test_pcp_manager_v2.py`

1. Covers persistent Ascend input buffers, prefix-sum offset shape,
padded local batches, main capture slot mappings, and old/new keyword
signatures.
2. Verifies `dp_sync` on all three Ascend speculators and `pcp_manager`
on graph capture.
- Upstream coverage:
[#53515](vllm-project/vllm#53515),
[#53694](vllm-project/vllm#53694), and
[#53869](vllm-project/vllm#53869).

### Does this PR introduce any user-facing change?

No. This is an internal compatibility update; it does not add an
Ascend-specific public API.

### How was this patch tested?

- Exact-contract main2main validation and range prediction for vLLM
[`ba07e4a48...e6bfe03ad`](vllm-project/vllm@ba07e4a...e6bfe03).
- GitHub Actions: [run
33229439657](https://github.com/vllm-project/vllm-ascend/actions/runs/33229439657).
- Successful in that run: pre-commit, both 310P jobs, all A3 jobs, and
the passing A2 shards on both `e6bfe03ad...` and `v0.27.1`.
- Pending rerun: ModelScope HTTP 500 failures on A2; two unchanged EPLB
CPU tests fail identically on both lanes.

- vLLM main:
vllm-project/vllm@ba07e4a

---------

Signed-off-by: liaoqidan <1107297340@qq.com>
Signed-off-by: shenzhao <shenzhao9@huawei.com>
Signed-off-by: LQDLove <LQDLove@users.noreply.github.com>
Co-authored-by: shenzhao <shenzhao9@huawei.com>
Co-authored-by: LQDLove <LQDLove@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working mrv2 Model Runner V2 specific nvidia ready ONLY add when PR is ready to merge/full CI is needed

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants